Emit gen_ai.client.token.usage and operation.duration metrics from the OTel middleware - #666
Conversation
026998e to
5773147
Compare
This comment has been minimized.
This comment has been minimized.
5773147 to
23dfb1a
Compare
github-actions
Bot
added
the
parity-approved
This comment has been minimized.
This comment has been minimized.
…el middleware
The OTel middleware recorded token counts only as span attributes, which are
not aggregatable across runs, so a spend/latency dashboard saw nothing from the
Go port. Add a Meter alongside the tracer and record the two histograms the
GenAI semantic conventions define and the Python/.NET SDKs emit:
gen_ai.client.token.usage ({token}) split by gen_ai.token.type input/output, and
gen_ai.client.operation.duration (s). Both are recorded on the normal and
early-return exit paths, tagged with operation/provider/agent name plus
error.type when the run faults. Histogram-construction failures leave a working
tracer-only middleware.
23dfb1a to
0076565
Compare
This comment has been minimized.
This comment has been minimized.
# Conflicts: # go.mod # provider/otelprovider/otel.go
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
Pull request overview
This PR extends the provider/otelprovider OpenTelemetry middleware to emit GenAI semantic-convention metrics in addition to existing span attributes, enabling cross-run aggregation for token usage and operation duration (dashboards/alerts) while preserving per-run span detail.
Changes:
- Add OTel Metrics instruments (
gen_ai.client.token.usage,gen_ai.client.operation.duration) to the existing tracing middleware and record them per run. - Capture and attach shared identifying attributes (and
error.typeon failures) to emitted metric data points. - Add black-box tests using an in-memory
sdkmetric.ManualReaderto assert emitted histograms and error attribution.
Reviewed changes
Copilot reviewed 3 out of 4 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| provider/otelprovider/otel.go | Adds Meter-based histograms and records token-usage + duration metrics from the middleware. |
| provider/otelprovider/otel_test.go | Adds metric-focused tests validating histogram emission and error tagging. |
| go.mod | Promotes OTel metric and sdk/metric dependencies to direct requirements. |
| go.sum | Records new module checksums introduced by metric dependencies. |
| attrs := []attribute.KeyValue{ | ||
| attribute.String(attrKeyOperationName, opInvoke), | ||
| attribute.String(attrKeyProviderName, cmp.Or(a.ProviderName(), "unknown")), | ||
| attribute.String(attrKeyAgentName, a.Name()), | ||
| } | ||
| if errorType != "" { | ||
| attrs = append(attrs, attribute.String(attrKeyErrorType, errorType)) | ||
| } |
|
PratikDhanave (@PratikDhanave) conflicts |
# Conflicts: # go.mod # go.sum
github-actions
Bot
added
area:provider
This comment has been minimized.
This comment has been minimized.
github-actions
Bot
added
failed-auto-risk
github-actions
Bot
added
pending-auto-risk
This comment has been minimized.
This comment has been minimized.
github-actions
Bot
added
pending-auto-risk
API Consistency ReviewScope: user-visible behavior (OTel metrics emission) Changed Go contract: Upstream evidence reviewed:
Result: one parity finding reported — explicit histogram bucket boundaries Finding: Missing explicit histogram bucket boundariesThe Python implementation explicitly sets # python/packages/core/agent_framework/observability.py:157–188
TOKEN_USAGE_BUCKET_BOUNDARIES: Final[tuple[float, ...]] = (
1, 4, 16, 64, 256, 1024, 4096, 16384, 65536, 262144, 1048576, 4194304, 16777216, 67108864,
)
OPERATION_DURATION_BUCKET_BOUNDARIES: Final[tuple[float, ...]] = (
0.01, 0.02, 0.04, 0.08, 0.16, 0.32, 0.64, 1.28, 2.56, 5.12, 10.24, 20.48, 40.96, 81.92,
)
# passed as:
meter.create_histogram(..., explicit_bucket_boundaries_advisory=TOKEN_USAGE_BUCKET_BOUNDARIES)The Go PR creates the histograms without equivalent bucket advisory options: // provider/otelprovider/otel.go — NewMiddleware
if h, err := meter.Float64Histogram(
metricTokenUsage,
metric.WithUnit("{token}"),
metric.WithDescription("Measures number of input and output tokens used."),
); err == nil {
m.tokenUsage = h
}Semantic impact: Omitting the advisory boundaries means the Go OTel SDK uses its own default exponential/explicit-boundary sequence, which differs from the Python advisory sequence. Cross-language dashboards that aggregate histogram data from Go and Python agents will see different bucket distributions for identical workloads, making percentile (p50/p95) and heatmap comparisons misleading across SDKs. The total count and sum remain comparable, but the key point of histogram alignment is bucket-level comparability. Suggested resolution: Pass // Token usage buckets (powers-of-4, matching OTel GenAI advisory)
var tokenUsageBuckets = []float64{1, 4, 16, 64, 256, 1024, 4096, 16384, 65536, 262144, 1048576, 4194304, 16777216, 67108864}
// Operation duration buckets (doubling sequence in seconds, matching OTel GenAI advisory)
var operationDurationBuckets = []float64{0.01, 0.02, 0.04, 0.08, 0.16, 0.32, 0.64, 1.28, 2.56, 5.12, 10.24, 20.48, 40.96, 81.92}Note: In the OTel Go SDK the advisory boundaries are typically applied at the Because a parity issue was found, the
|
github-actions
Bot
removed
the
parity-approved
What
The OTel middleware previously recorded token counts purely as span attributes (
gen_ai.usage.*) viasetUsage, and never measured run duration at all. It imported onlyotel/{attribute,codes,trace}and held a lonetracer— no Meter, no histograms.Span attributes describe a single run and are not aggregatable across runs, so a spend/latency dashboard summing across many invocations saw nothing from the Go port. This change adds a Meter alongside the tracer and emits the two histograms the OTel GenAI semantic conventions define:
gen_ai.client.token.usage(unit{token}) — two data points per run, one taggedgen_ai.token.type=input(InputTokenCount) and oneoutput(OutputTokenCount).gen_ai.client.operation.duration(units) — one sample per run, measured from beforespan.Start.Every data point carries the run's identifying attributes (
gen_ai.operation.name,gen_ai.provider.name,gen_ai.agent.name), pluserror.typewhen the run faulted. Recording is factored into arecordMetricshelper (mirroringsetUsage) shared by both the normal post-loop path and the early-return path. Histogram construction errors are tolerated:NewMiddlewarestill returns a working tracer-only middleware andrecordMetricsno-ops on a nil histogram.Why (cross-SDK parity)
This aligns the Go port with the OTel GenAI semantic conventions and the Python/.NET references, which both emit these two histograms. Cross-language dashboards can now aggregate Go runs alongside the rest. Span-attribute accounting stays exactly as-is; metrics complement it (single-run detail vs. cross-run aggregation) rather than replacing it. Not a duplicate of the span PRs (#655/#646) or usage-accounting PRs (#594/#609/#551-554).
Tests
provider/otelprovider/otel_test.gogains two black-box tests driving an in-memorysdkmetric.ManualReader:TestOtel_Run_RecordsUsageAndDurationMetrics— runs the middleware over a fakeRunFuncyieldingUsageContent(100 input / 50 output), collectsResourceMetrics, and asserts the twogen_ai.client.token.usagedata points (input=100, output=50 bygen_ai.token.type) with the shared attributes, plus onegen_ai.client.operation.durationsample.TestOtel_Run_MetricsIncludeErrorType— assertserror.typeis present on the duration metric when the run faults.Both fail before the change and pass after.
go build ./...,go vet ./provider/otelprovider/..., andgo test ./provider/otelprovider/...are green.Open design questions
MiddlewareConfigis unchanged — the Meter reusesSourceNamefor its instrumentation scope, matching the tracer. Open to a separate meter name if preferred.AdditionalCountsand cache/reasoning sub-counters are not yet emitted as metric data points (the conventions only define input/output token-type dimensions); could be added if there's appetite.